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ABSTRACT 


A 

This  exploratory  study  investigates  how  writers  represent  their  task  to 
themselves  before  beginning  to  write.  Using  data  from  verbal  protocols,  we 
examine  the  initial  plans  of  twelve  writers  (five  experts  and  seven  student  writers) 
who  were  working  on  an  expository  writing  task.  The  protocols  were  coded  for 
types  of  planning.  We  also  obtained  independent  measures  of  the  quality  of  the 
subjects'  plans  and  of  the  quality  of  their  texts. 

The  analysis  suggests  that  both  the  quantity  and  quality  of  a  writer's  initial 
planning  may  make  a  difference  in  the  quality  of  the  final  text.  We  found  a  positive 
correlation  between  the  amount  of  initial  planning  and  text  quality,  and  between  the 
quality  of  planning  and  text  quality.  In  particular,  we  found  that  writers  who 
developed  rhetorical  plans  (i.e.,  plans  for  audience  and  purpose)  tended  to  produce 
higher-rated  texts.  From  our  analysis,  we  hypothesize  that  experienced  writers 
build  a  rhetorical  representation  of  their  task.  We  defined  a  rhetorical  representation 
as  one  which  is  rich  in  rhetorical  goals  and  plans  relating  to  the  audience,  purpose, 
form  and  language  of  the  text,  and  in  which  the  writer  integrates  his  plans  to  form  a 
coherent  theory  of  the  task,  j 
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I.  INTRODUCTION 


Writing  problems  fall  into  the  general  class  of  problems  characterized  by 
Reitman  (1964)  and  Simon  (1973)  as  "ill- structured"  problems,  analogous  to 
problems  faced  by  architects,  designers  or  composers.  In  contrast  to 
"well-structured"  problems  (e.g.,  games,  or  toy  tasks)  where  the  goals,  operators 
and  knowledge  required  for  solving  the  problem  are  specified  for  solvers  by  the 
task  instructions,  ill-defined  problems  require  solvers  to  define  their  own  "problem 
space"  as  they  work  (Simon,  1973).  Dealing  with  an  ill-structured  problem,  then, 
requires  problem-solvers  to  take  an  active  role  in  delimiting  and  defining  the 
boundaries  of  their  task,  and  more  specifically  to: 

•  Build  their  own  unique  representation  of  their  task  in  response  to  some 
general  (and  often  vague)  task  specifications— e.g.,  "compose  a  fugue," 
"write  about  your  job." 

•  Specify  their  own  goals  and  criteria  for  the  task  and  develop  their  own 
tests  for  evaluating  possible  solutions. 


While  falling  within  the  general  category  of  ill-defined  problems,  writing 
tasks  may  vary  in  the  amount  of  structure  imposed  by  the  task  instn’Ctions.  In 
some  tasks,  like  writing  a  computer  manual  to  tight  specifications,  some  constraints 
(such  as  format  and  style)  may  be  given  to  the  writer.  In  many  others,  however, 
writers  are  given  a  "clean  slate"  to  define  their  own  goals  and  criteria.  In  such 
cases,  a  writer  is  faced  with  a  myriad  of  choices:  there  may  be  many  possible  ways 
to  represent  the  task  and  problem  and  many  possible  solutions-i.e.,  types  of  text 
that  might  be  produced.  The  choices  that  writers  make  (or  do  not  make)  in  terms  of 
goals,  constraints  (such  as  genre)  and  criteria  provide  a  self-imposed  structure  to 
the  task  and  guide  their  search  for  possible  solutions;  writers  can  only  accomplish 
the  tasks  which  they  set  themselves  (Flower  et  al.,  1987). 

How  then  do  writers  represent  their  tasks  to  themselves?  What  are  the 
features  of  their  representations  and  what  individual  differences  might  we  find? 
Here,  we  will  report  the  results  of  an  exploratory  study  of  the  initial  task 
representations  of  twelve  writers  (with  varying  amounts  of  writing  expertise)  for  a 
complex  writing  task.  [This  research  was  part  of  a  major  study  of  experts'  and 
novices'  planning  processes  in  which  we  develop  a  theory  of  the  planning  process 
and  examine  writers'  goals  and  plans  before  and  during  composing.  See  Flower 
et  al.,  1987;  Hayes,  1987].  Before  discussing  our  study  of  initial  task 
representation  in  depth,  let  us  look  at  some  findings  from  other  domains  which 
provide  some  working  hypotheses  about  the  nature  of  and  differences  in 
problem-solvers'  representations  of  their  task. 

Research  in  other  knowledge-rich  domains  points  to  some  important 
differences  between  experts'  and  novices'  problem  representations.  For  a  case  in 
point,  let  us  examine  solvers'  representations  of  problems  in  physics.  Although 
physics  can  be  classified  as  a  "formal  domain"  (Larkin,  1983)  where  the  goals  and 
operators  are  more  well-defined  than  in  most  writing  tasks,  we  can  see  some 
important  analogies  with  writing;  there  are  several  possible  problem  representations 
that  solvers  may  develop,  and  the  choice  of  representation  greatly  affects  both  the 
efficiency  of  their  search  for  a  solution  and  the  quality  of  that  solution.  For 
example,  Larkin  (1983)  and  Larkin,  McDermott,  Simon  &  Simon  (1980)  found  that 
physics  experts  build  representations  of  physics  problems  which  are  qualitatively 
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different  from  those  of  physics  novices.  First,  the  experts  built  "elaborated" 
representations  which  contained  a  large  amount  of  domain  knowledge;  second,  their 
representations  were  based  on  an  inter-ielated  set  of  physics  principles  which 
Simon  and  Simon  (1978)  term  "physical  intuition."  In  contrast  the  novices' 
representations  were  much  more  spare  and  were  based  on  formulae  for  solving  the 
problem  rather  than  on  physical  principles  and  concepts.  As  a  result,  the  novices 
often  picked  up  on  cues  in  the  task  instructions  and  jumped  straight  into  the 
problem  without  conceptualizing  it;  sometimes  they  were  mistaken  in  their  choice  of 
formula,  and  because  they  did  not  represent  the  problem  on  a  more  fundamental 
level,  they  were  unable  to  proceed  to  a  solution. 

This  research  on  physics  solvers  suggests  some  interesting  questions  about 
writers'  initial  task  representations: 

•  Do  writers'  task  representations  also  show  qualitative  differences 
depending  on  the  writer's  expertise  and  experience?  How  can  we 
characterize  these  differences? 

•  Do  expert  writers  build  enriched  task  representations  based  on  rhetorical 
principles  and  relationships— in  essence,  representations  based  on  what 
we  might  term  "rhetorical  intuition"? 

•  If  so,  do  such  "rhetorical  representations"  function  in  a  similar  way  to 
experts'  "physical  representations"— that  is,  to  allow  expert  writers  to 
structure  their  task  in  a  way  conducive  to  a  successful  solution? 

We  envisage  that  a  rhetorical  representation  may  provide  an  exj^rt  writer 
with  an  overall  frameworK  for  his  or  her  planning  process.  Thus  effective  writers 
may  develop  plans  which  encompass  important  rhetorical  concepts  which  they 
instantiate  with  goals  and  plans  unique  to  the  task  at  hand.  That  is,  such  a  rhetorical 
representation  would  call  for  the  writer  to  include  goals  and  plans  not  only  for  the 
content  of  the  text,  but  also  for  its  organization,  for  its  overall  focus,  and  for 
meeting  the  needs  of  its  particular  audience  and  to  integrate  these  components.  Thus 
we  would  expect  to  see  a  writer  planning  a  text  around  these  goals  and  embedding 
content  within  a  purpose. 

This  kind  of  rhetorical  representation  requires  more  than  a  "fill  in  the  blank" 
approach  to  a  writing  task.  Each  new  task  requires  writers  to  instantiate  their 
top-level  goals  (e.g.,  to  meet  the  needs  of  the  reader)  in  a  different  way.  If  the 
subject  is  complex  and  new,  writers  may  not  have  available  pre-packaged  or 
organized  pieces  of  information  which  can  be  simply  slotted  into  the  text.  Rather, 
they  will  have  to  engage  in  difficult  knowledge  transforming  operations  to  adapt 
what  they  know  to  meet  their  rhetorical  goals  (Scardamalia  &  Bereiter,  in  press). 
We  would  certainly  expect  more  experienced  writers  to  be  more  adept  at  this 
difficult  manipulation  of  content  and  rhetorical  knowledge.  For  example,  in  a  study 
of  young  writers,  Burris  et  al.  (1983)  found  that  their  subjects'  plans  consisted 
overwhelmingly  of  plans  for  content  and  that,  even  when  provided  with  cues  for 
more  rhetorically  oriented  planning,  the  subjects  still  produced  plans  for  specific 
content  or  for  actual  text. 

Our  analysis  has  led  us  to  identify  five  key  features  of  a  rhetorical  task 
representation.  We  hypothesize  that  writers  who  engage  in  initial  rhetorical 
planning  develop  plans  in  the  following  categories.  (The  examples  presented  here 
are  from  verbal  protocols  collected  from  writers  working  on  our  experimental 
task— to  write  about  their  job  for  the  teenage  magazine.  Seventeen.  We  discuss  our 
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task,  subjects  and  procedures  more  fully  in  the  methodology  section  of  this  report. 
We  also  present  some  extended  episodes  from  our  verbal  protocols,  with  an 
analysis  of  their  planning  statements,  later  in  the  report--see  Section  III.) 


•  Content;  What  information  will  the  writer  include  in  this  text? 

e.g.  So  maybe  I  could  write  about  a  college  student 
preparing  for  my  major  or  a  job  which  would  be 
public  policy  and  management 

•  Form:  How  will  the  writer  organize  and  structure  this  information? 

What  genre  or  format  will  he  or  she  use? 

e.g.  What  about  a  sequence.  Reads  papers,  makes  up 
assignments,  writes— so  forth.  I  want  to  start  with 
what  they  think  is  obvious 

•  Audience:  How  does  the  writer  represent  the  audience  (for  example, 

their  characteristics,  attitudes,  interests  etc.)? 

e.g.  Young  female  teenage  audience.  They  will  have 
had  English— audience-they're  in  school 

•  Theme:  How  does  the  writer  represent  the  overall  focus  or  rhetorical 

purpose  of  the  text?  Does  the  writer  have  a  unifying  idea 
around  which  other  ideas  are  developed? 

e.g.  In  fact  that  might  be  a  useful  thing  to  focus  on. 
How  a  teacher  differs  from  a  professor  and  I  see 
myself  as  a  teacher 

•  Other  goals:  What  other  goals  does  the  writer  set  for  this  task?  For 
example,  what  goals  does  the  writer  have  for  addressing  the  audience, 
for  projecting  his  or  her  own  persona,  for  the  choice  of  language  and 
tone? 


e.g.  The  tone  and  style  at  seventeen.  I  suppose  it  should 
be  light  and  lively,  filled  with  slang 


What  alternatives  to  a  rhetorical  representations  might  a  writer  use?  First, 
inexperienced  writers  may  organize  their  writing  tasks  around  content  rather  than 
rhetorical  features;  that  is,  they  set  themselves  the  goal  to  "tell  all  they  know"  about 
a  topic  rather  than  to  adapt  or  transform  their  knowledge  to  meet  the  rhetorical 
constraints  of  audience  and  purpose  (Scardamalia  &  Bereiter,  1982;  Scardamalia  & 
Bereiter,  in  press).  This  "knowledge-telling"  strategy  may  be  a  useful  way  for 
young  writers  to  express  their  thoughts,  but  it  is  clearly  inappropriate  for  complex 
writing  tasks.  This  distinction  is  similar  to  that  (for  adult  writers)  made  by  Kern 
et  al.  (1977)  between  topic-oriented  writing  and  performance-oriented  writing;  as 
Kern's  study  indicates,  where  a  text  will  be  used  to  facilitate  action  or 
decision-making  (for  example,  instructions  for  operating  a  machine  or  policy 
guidelines),  topic-oriented  writing  will  not  meet  the  needs  of  the  user/reader. 
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Second,  as  we  noted  earlier,  not  all  writing  tasks  may  require  a  sophisticated 
rhetorical  task  representation.  For  example,  take  the  case  of  a  journalist  writing  a 
news  story.  In  such  a  highly  structured  genre,  a  standard  format— for  example,  the 
"inverted  pyramid"  which  requires  an  ordering  of  information  according  to 
importance— provides  a  ready-made  set  of  genre-specific  goals,  constraints  and 
criteria  for  the  task  (Schumacher,  Klare  and  Scott,  1985).  In  such  tasks,  expert 
writers  may  not  need  to  build  a  unique  and  "customized"  representation  of  the  task 
because  they  already  have  a  well-learned,  "off  the  shelf  script  which  guides  and 
constrains  their  writing  process.  Their  problem-solving  efforts,  then,  may  be 
directed  to  other  cognitive  activities,  such  as  evaluating  the  extent  to  which  their 
emerging  text  meets  the  genre  constraints  (Schumacher  et  al.,  1985). 

The  goals  of  our  study  of  initial  task  representation  were,  then: 

•  To  investigate  possible  differences  in  the  type  of  initial  task 
representation  our  writers  built.  In  particular,  we  wanted  to  see  if  our 
writers  built  rhetorically-based  representations  and  what  kind  of 
information  they  included. 

•  To  investigate  whether  the  type  and  quality  of  the  initial  plans  that 
writers  m^e  may  be  related  to  the  quality  of  their  final  text;  that  is, 
wnether  those  writers  whose  plans  had  a  more  rhetorical  basis 
produced  better  texts. 

We  wanted  to  leave  open  (as  far  as  possible)  the  goals,  constraints  and 
criteria  for  the  task  to  enable  writers  to  build  a  rhetorical  representation  if  they  were 
able,  or  if  they  chose,  to  do  so.  We  thus  chose  a  task  which  was  complex  and 
new,  and  which  would  then  not  lend  itself  to  either  a  knowledge-telling  task 
representation  or  a  formula-based  representation. 

II.  METHODOLOGY 


Subjects 

The  subjects  for  this  study  were  twelve  writers  with  varying  degrees  of 
expertise  and  experience.  Five  were  expert  writing  teachers  and  seven  were  student 
writers  (four  advanced  students  and  three  with  identified  writing  difficulties). 

Task 


The  subjects  were  asked  to  write  about  their  job  for  Seventeen  Magazine. 
They  were  told  that  the  readers  of  the  magazine  would  be  teenage  girls  (aged 
thirteen  or  fourteen).  The  subjects  were  given  an  hour  to  complete  their  task,  and 
verbal  protocols  were  taken  as  they  worked. 

This  task  was  ill-defined  in  that  each  writer  could  draw  on  and  adapt  a 
unique  body  of  personal  knowledge  about  his  or  her  job  and  develop  a  unique  set 
of  goals  to  reach  the  young  readers  of  the  article.  Clearly,  there  was  a  variety  of 
ways  of  approaching  this  task  and  hence  a  potential  for  unique,  adaptive  solutions 
to  a  complex  rhetorical  problem.  Our  writers  on  this  task  had  to  do  what  many 
professionals  writing  to  lay  audiences  have  to  do:  1)  determine  their  own  goals, 
including  representing  the  audience  to  themselves  and  deciding  how  to  meet  the 
audience's  needs;  and  2)  decide  what  information  to  include,  given  their  goals. 

Even  the  topic  of  "their  job"  was  open  to  debate  and  depended  in  part  on  tiieir  image 
of  the  audience  and  the  goals  they  set  for  the  essay. 
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Method  of  Analysis 

1.  Text  Rating 

The  subjects'  written  products  were  rated  for  quality  by  four  experienced 
writers  who  considered  the  following  three  dimensions  of  the  essays: 

•  How  well  is  the  text  adapted  for  the  audience? 

•  To  what  extent  does  this  text  have  a  clear  point,  focus  or  rhetorical 
purpose  that  goes  beyond  simply  "knowledge-telling"  on  a  topic? 

•  How  well-structured  is  the  text  in  terms  of  overall  organization  and 
coherence? 

(See  Appendix  A  for  the  rating  instructions.) 

We  chose  these  three  criteria  because  they  address  important  rhetorical 
features  of  many  professional  writing  tasks  (for  example  in  the  our  task,  adapting 
information  about  one's  job  for  young  readers  who  might  not  be  very  motivated  to 
read  on).  We  wanted  to  obtain  a  measure  of  the  extent  to  which  our  writers  met  the 
rhetorical  constraints  of  the  task.  Thus,  because  our  criteria  forjudging  a  text  as 
"successful"  were  rhetorical,  rather  than  purely  based  on  content,  we  did  not 
explicitly  ask  our  raters  to  evaluate  the  content  of  the  essays  in  isolation  from  these 
rhetorical  features.  For  example,  consider  a  writer  who  described  the  technical 
aspects  of  his  job  as  a  computer  technician  without  adapting  for  the  audience  (e.g., 
without  explaining  technical  concepts  and  terminology)  or  without  manifesting  a 
clear  rhetorical  purpose  other  than  "knowledge-telling";  this  writer  would  receive  a 
relatively  low  total  score  for  text  quality,  even  if  he  presented  a  lot  of  detailed  and 
accurate  information,  because  his  text  would  not  meet  our  rhetorical  criteria. 

2.  Analysis  of  Planning  Categories  in  the  Protocols 

For  this  study  of  initial  planning,  we  limited  our  analysis  of  the  protocols  to 
the  initial  segments,  which  ended  when  ti\e  subjects  wrote  their  first  complete 
sentence;  we  did  not  look  at  any  planning  that  they  did  after  this  point.!  This  unit 
of  the  protocols  provides  us  with  a  conservative  measure  of  the  subjects'  planning 
ability  in  that  it  only  provides  information  on  how  our  writers  represented  their  task 
to  themselves  b^ore  beginning  to  actually  generate  a  solution  in  the  form  of  text. 
We  argued  that,  although  some  features  of  their  task  representation  may  have  been 
modified  in  response  to  evaluations  of  their  emerging  text,  writers  may  set  the 
boundaries  of  their  task  in  this  initial  segment— that  is,  they  may  set  their  top-level 
goals  and  constraints.  (The  average  length  for  this  segment  was  70  clause-units.) 
Would  differences  in  the  nature  of  the  task  that  our  writers  set  themselves  in  this 
initial  planning  account  for  differences  in  the  nature  of  the  texts  they  produced? 

The  protocols  were  analyzed  as  follows:^ 

1 .  The  protocols  were  divided  into  clauses  up  to  the  cut-off  point  defined 
above,  and  clause-units  were  coded  into  the  following  categories: 
reading/paraphrasing  the  task  instructions;  process  goals  and  comments; 
metacomments  relating  to  the  assignment;  planning.  We  will  focus  on 
those  statements  coded  as  "planning." 
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2.  These  planning  clauses  were  further  broken  down  into  the  five 

categories  which  we  hypothesized  constitute  key  features  or  categories 
of  planning  in  a  rhetorical  task  representation;  content/topic,  form, 
au^ence,  theme,  and  other  goals  (see  discussion  and  examples  in 
Section  1).  Clauses  which  were  simply  repetitions  of  information 
previously  stated  were  excluded  from  this  analysis. 

3.  Rating  of  Planning  Quality  in  the  Protocols 

In  addition  to  rating  the  texts,  we  obtained  a  measure  of  the  quality  of  the 
planning  in  the  initial  planning  segments  of  the  protocols.  Two  experienced  writers 
rated  the  protocol  excerpts  on  the  three  dimensions  listed  below.  The  raters  were 
explicitly  told  to  consider  quality  of  planning,  rather  than  amount  of  planning  in 
these  categories.  (See  Appendix  A  for  the  rating  instructions.) 

•  How  well  does  the  writer's  planning  reflect  a  concern  for  adapting  his 
or  her  text  for  the  audience? 

•  To  what  extent  does  the  writer's  planning  reflect  a  concern  with 
developing  a  clear  point,  focus  or  rhetorical  purpose  for  his  or  her  text 
that  goes  beyond  simply  "knowledge-telling"  on  a  topic? 

•  How  far  does  the  writer's  planning  reflect  a  concern  with  structuring  the 
text  or  fitting  a  genre? 

III.  RESULTS  AND  DISCUSSION 

Here  we  will  look  at  two  aspects  of  our  subjects'  planning  in  these  initial 
segments  of  the  protocols:  1)  the  amount  of  planning  that  the  subjects  engaged  in 
before  beginning  to  write  and  2)  the  nature  and  quality  of  their  planning.  In  our 
analysis,  we  categorize  our  subjects  in  several  different  ways: 


Category 

Breakdown  of  Subjects 

Previous  writing  experience: 

Expert 

Novice 

Amount  of  initial  planning: 

Based  on  number  of  planning  clauses 
up  to  first  sentence  (median  split) 

Extensive 

Planners 

Minimal 

Planners 

Quality  of  fmal  text: 

Based  on  total  quality  scores  for 
text  (median  split) 

Higher 

Scorers 

Lower 

Scorers 

Type  of  task  representation: 

Based  on  %  of  content  plans  vs. 
rhetorical  plans  and  categories  of 
rhetorical  plans  covered 

Rhetorical 

Content-based 

CW'e  will  discuss  these  taicgories  in  more  detail  as  they  ai  ise  in  oui  analysis.) 
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We  expected  that  the  above  four  dimensions  would  be  related— that  is,  that  our 
expert  writers  would  produce  texts  in  the  top  half  of  the  distribution  of  text  scores 
(high  scorers)  and  that  they  would  develop  extensive,  rhetorical  plans  in  their 
initial  task  representations.  While  two  experts  did  fit  this  profile,  three  others  did  not 
(they  were  missing  one  or  more  dimensions).  Similarly,  not  all  our  novices  were 
"lower  scorers";  several  of  them  were  extensive  planners^  and  some  did  engage  in 
rhetorical  planning  (though  perhaps  not  with  the  sophistication  of  our  expert 
rhetorical  planners).  These  somewhat  surprising  results  are  discussed  further  in  the 
sections  which  follow. 

Amount  of  Initial  Planning 

We  found  a  wide  range  in  the  amount  of  planning  that  the  subjects  engaged  in 
before  beginning  to  write  (see  Table  1).  However,  counter  to  what  one  might  expect, 
the  amount  of  initial  planning  did  not  correspond  with  experience.  (There  was  not  a 
significant  difference  between  the  number  of  planning  statements  for  experts  and 
novices  by  the  Mann-Whitney  test.)  The  writers  who  did  little  planning  were  not 
always  the  novices;  for  example,  two  of  our  experts  (BS  and  IB)  have  only  7  and  2 
planning  statements  respectively.  Rather,  we  found  two  modes  of  planning^  which 
cut  across  expertise:  "extensive,"  where  the  subjects  do  a  substanti^  amount  of 
planning  and  "minimal,"  where  the  subjects  do  little,  or  almost  no  planning. 

This  numerical  distinction  also  appears  to  reflect  two  distinct  approaches  to 
initial  planning  which  we  observed  in  the  protocols.  Our  six  minimal  planners 
generally  took  an  approach  which  can  be  characterized  as  "plunge  in  and  write," 
whereas  our  six  extensive  planners  did  try  to  develop  some  initial  plans,  with  varying 
de^ees  of  success.  To  illustrate  this  distinction,  we  present  below  examples  of  the 
initial  task  representations  of  two  of  our  typical  "minimal"  planners-one  expert  and 
one  novice.  Neither  of  these  subjects  did  any  initial  planning  that  went  beyond 
briefly  stating  the  overall  topic  for  their  text,  and  (in  the  case  of  the  novice)  making 
brief  references  to  the  audience;  both  subjects  made  a  decision  to  begin  writing  early 
in  their  composing  process.  The  expert,  in  fact,  consciously  chose  to  write  in  order 
to  "see  what  (he  will)  come  up  with";  and  he  subsequently  only  made  minor  changes 
to  the  draft  that  he  began  at  the  end  of  this  initial  episode. 

Minimal  Planner:  Expert  IB  (Total  planning  =  2cls.) 

(Reads  and  paraphrases  assignment).  "Okay— let's  see..NEH  fellow.  What's 
NEH.  Okay  explain  NEH-going  to  have  to  do  an  awful  lot  of  explaining. 
Don't  forget  the  audience.  Okay  I'm  going  to  try  a  little  prewriting  to  see  what 
I  come  up  with  here...  NEH..um.  Alright..charge  ahead 
here..charge  ahead  and  see  what  happens.  I'm  an  NEH  fellow  at 
CMU—  No,  job,  job,  my  job.  Let's  try  another  tack  here" 

(Begins  to  write  a  draft) 

Minimal  Planner:  Novice  FD  (total  planning  =11  els.) 

(Reads  assignment).  "I  don't  know  where  to  begin.  I  don't  have  any 
thoughts  yet.  What  do  they  want  to  hear.. .I'm  talking  about  these  girls..  I 
don't  see  why  I'm  going  to  write  different  for  them  than  I  would  (for) 
everybody  else.  When  they  say  my  job  what  do  they  mean?  Presently  or 
future,  one  you  want  to  hold  or  a  job  you've  had  in  the  past?  Maybe  a  job  I 
want  to  have..  I  work  in  the  computer  science  room  at  CMU.  That's  where  I 
want  to  work...  Maybe  I'll  take  to  the  prose  and  try  to  write  to  these 
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kids  about  what  they  should  look  for  in  a  job.  First  of  all,  the  job  I  want  to 
work  in  is  in  a  business-related  environment  of  the  computer  science  field.  If 
possible  the  department  of  defense..  I'm  going  to  try  to  write  something  that 
they're  going  to  want  to  read.. .and  just  start  saying  I'm  basically  looking  for 
a  job."  (Begins  to  write  a  draft) 

In  contrast,  as  we  can  see  from  Table  1,  our  extensive  planners  delayed 
writing  a  first  sentence  until  they  had  worked  longer  on  the  task.  All  these  writers 
were  at  least  attempting  to  develop  some  more  sub^stantial  initial  plans  for  their  text 
(although,  there  was  a  large  variability  in  the  number  of  planning  clauses  excluding 
repetitions  in  the  initial  task  representations  of  the  "extensive  planners":  lowest  =  21; 
highest  =  88).  Let  us  consider  the  case  of  novice  writer  LR  (see  Table  1)  who  did  the 
least  amount  of  initial  planning  among  our  extensive  planners  (21  clauses).  In 
comparison  with  the  minimal  planners  discussed  above,  writer  LR  spent  more  time 
developing  plans  for  the  content  of  his  essay  and  was  thus,  we  would  argue,  at  least 
attempting  initial  planning.  (In  Section  IB,  we  present  a  transcript  of  this  writer's 
initial  task  representation  and  discuss  his  lack  of  success  in  developing  many 
rhetorical  plans-e.g.,  plans  for  audience  and  purpose-in  more  depth.) 

Amount  of  Initial  Planning  and  Text  Quality 

We  found  the  amount  of  our  writers'  initial  planning  to  be  positively 
correlated  with  the  quality  of  the  text  they  produced.  (See  Table  2  for  text  scores.)^ 
Overall,  we  found  a  correlation  of  .655  between  the  number  of  planning  statements 
made  by  our  writers  and  the  total  quality  score  for  the  written  texts.  For  the  experts 
the  correlation  was  .694  and  for  the  novices,  .799.  (These  figures  were  computed 
using  the  Pearson  product  moment  correlation  co-efficient.)  The  high  correlation 
between  the  novices'  planning  and  the  quality  of  their  texts  is  particularly  interesting. 
The  fact  that  the  two  novices  who  did  the  least  planning  (MR  with  4  cl.  and  GJ  with  8 
cl.)  received  the  lowest  scores  for  their  texts  and  that  two  of  the  novices  who  did  the 
most  planning  (NK  with  44  cl.  and  SA  with  38  cl.)  received  the  highest  scores  for 
their  texts  (out  of  the  novice  writers)  suggests  that  the  amount  of  planning  before 
beginning  to  write  may  be  a  good  predictor  of  the  quality  of  the  text  produced  when 
an  inexperienced  writer  is  facing  a  relatively  complex  task. 

We  hypothesize  that  a  key  variable  in  predicting  success  on  our  task  may  have 
been  the  amount  of  initial  planning  rather  than  prior  writing  experience.^  We  found 
no  significant  differences  (by  the  Mann- Whitney  test)  between  the  quality  of  the  texts 
produced  by  our  expert  and  our  novice  writers.  Although,  the  difference  between  the 
quality  of  the  texts  produced  by  the  extensive  planners  and  by  the  minimal  planners 
(see  Table  2)  was  also  not  statistically  significant,  though  it  approached  significance 
(p  =.0782  by  the  Mann- Whitney  test).  However,  the  positive  correlation  between 
amount  of  initial  planning  and  text  quality  (and  especially  the  high  correlation  for 
novice  writers  of  .799)  does  suggest  a  relationship  which  warrants  further 
investigation. 

Although  these  results  are  exploratory,  they  do  suggest  that  building  an  initial 
task  representation  can  positively  affect  the  quality  of  the  texts  produced.  The 
importance  of  planning  throughout  a  writing  task  has  been  observed  in  previous 
studies  (see  Flower  and  Hayes,  1981).  This  current  study  suggests  that  initial 
planning  is  one  aspect  of  the  planning  process  which  contributes  to  producing  a 
successful  text,  particularly  for  inexperienced  writers.  This  hypothesis  is  certainly 
consistent  with  findings  from  other  domains  (such  as  physics)  where  novices' 
difficulties  often  result  from  a  lack  of  time  spent  conceptualizing  the  problem  before 
beginning  to  work.  The  correlation  between  amount  of  initial  planning  and  text 
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"^oality  for  our  experts  is  more  surprising,  however.  We  might  expect  that 
experienced  writers  would  produce  a  successful  text  with  little  or  no  initial  planning 
because  they  could  draw  on  well-developed  schemas  or  familiar  genres  to  set  their 
initial  goals.  However,  as  we  noted  earlier,  only  one  of  our  expert  writers  who  did 
"minimal"  initial  planning  received  a  score  in  the  top-half  of  the  distribution  of  text 
quality  scores  (see  Table  2;  expert  BS).  One  explanation  may  lie  in  the  ill-defined 
nature  of  our  task  which  demanded  a  unique  and  adaptive  set  of  top-level  goals. 

The  Quality  and  Nature  of  the  Initial  Task  Representations 

We  felt  that  quantity  of  initial  planning  alone  was  too  gross  a  measure  of  a 
writer’s  ability  to  build  an  effective  initial  task  representation;  we  were  also  concerned 
with  the  quality  of  the  information  in  these  initial  planning  episodes.  Thus,  in 
addition  to  obtaining  ratings  on  the  quality  of  our  subjects'  final  texts,  we  also 
obtained  judgments  from  different  raters  on  the  quality  of  their  initial  plans  (up  to 
their  generation  of  a  first  sentence).  That  is,  the  raters  looked  at  the  quality  of  the 
planning  statements  that  writers  made  during  the  initial  segment  of  the  protocol  in 
terms  of  whether  the  subjects  made  appropriate  plans  for  addressing  the  audience, 
whether  they  developed  a  clear  overall  rhetorical  purpose  and  whether  they  developed 
a  clear,  coherent  structure.  Note  that  here  the  raters  were  not  asked  to  consider  the 
amount  of  planning,  but  the  nature  of  the  rhetorical  information  in  a  subjects'  plans. 
The  ratings  for  qu^ity  of  planning^  are  presented  in  Table  3. 

We  found  no  significant  expert/novice  difference  in  the  quality  of  the 
planning;  however,  those  writers  who  did  more  planning  (that  is,  the  "extensive" 
planners)  obtained  significantly  higher  total  quality  scores  for  their  planning  than  did 
the  "minimal"  planners  (p  =  .0374  by  the  Mann- Whitney  test).  Although  we  were 
measuring  quality  of  planning  rather  than  quantity,  it  is  not  very  surprising  that  in 
most  cases  the  extensive  planners  received  higher  quality  scores  than  the  minimal 
planners  simply  because  they  elaborated  on  their  goals  more  fully;  however,  it  is 
interesting  to  note  that  two  extensive  planners  (expert  KC  and  novice  LR;  see 
Table  3)  did  receive  quality  scores  at  or  below  the  median  quality  score,  which 
suggests  that  although  they  did  some  extended  planning,  their  plans  were  not  very 
rich  in  rhetorical  information. 

An  interesting  result  of  this  analysis  was  the  very  high  positive  correlation 
between  the  quality  of  the  planning  and  the  quality  of  the  final  texts  (.874  by  the 
Pearson  product  moment  correlation  co-efficient).  Thus  the  writers  who  were  judged 
by  one  set  of  raters  to  have  the  plans  most  rich  in  information  were  judged 
independently,  by  a  different  set  of  raters,  to  have  the  best  texts.  This  result  suggests 
that  the  type  and  quality  of  a  writer's  planning  may  well  influence  the  quality  of  the 
text  that  he  or  she  produces.  The  sub-scores  for  quality  of  planning  for  the  audience, 
purpose  and  structure  showed  a  positive  correlation  with  the  sub-scores  for  these 
dimensions  of  text  quality  as  follows:  audience  =  .778;  purpose  =  .741;  structure  = 
.489.  It  would  seem  from  these  results,  then,  that  concern  with  audience  and  purpose 
in  planning  a  text  may  well  result  in  texts  which  are  better  on  these  two  dimensions. 

Taken  along  with  the  strong  correlation  between  quantity  of  planning  and  text 
quality  discussed  in  the  previous  section,  this  relationship  between  quality  of  initial 
planning  and  text  quality  suggesii  that,  for  many  writers,  the  planning  which  goes 
into  building  an  initial  representation  of  the  task  pays  off.  Although  this  hypothesis 
requires  further  testing,  we  feel  that  this  exploratory  analysis  at  least  underscores  the 
importance  for  writers  of  what  we  have  termed  initial  "constructive  planning." 
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Types  of  Initial  Planning 

Why  does  planning  seem  to  help  the  quality  of  the  texts  produced?  In  looking 
for  evidence  of  the  "rhetorical  planning"  discussed  earlier,  we  found  differences  in 
the  kinds  of  information  embedded  in  the  planning  statements.  Table  4  shows  the 
types  of  planning  information^  generated  by  higher  and  lower  scorers.  (The  scores 
are  based  on  qudity  scores  for  the  text;  see  Table  2.)  Some  writers  whom  we  had 
designated  as  "experts"  because  they  were  writing  teachers  received  relatively  low 
scores  for  their  texts  and  were  thus  placed  in  the  "low  scorers"  category  for  ^is 
analysis;  conversely  some  student  writers,  originally  designated  as  "novices," 
received  scores  in  the  top  half  of  the  distribution  of  scores.^  One  explanation  for  this 
seemingly  anomalous  result  may  be  that  some  of  the  student  writers  were  more  easily 
able  to  relate  to  the  interests  of  their  young  readers  and  thus  produced  more  engaging 
texts  than  some  of  the  writing  teachers. 

First,  we  looked  at  whether  our  writers  focused  on  topic  (content)  planning  or 
on  the  more  rhetorical  aspects  of  planning,  such  as  audience  and  purpose.  We  found 
that  in  terms  of  absolute  numbers  our  higher  scorers  simply  did  more  planning  in  all 
categories,  and  particularly  in  the  topic  category  (high  scorers'  total  topic  planning 
statements  =  159;  low  scorers  =  47).  Although  the  difference  in  the  number  of 
planning  statements  for  high  and  low  scorers  was  not  statistically  significant,  we  did 
find  a  positive  correlation  between  the  absolute  number  of  topic  planning  statements 
and  text  quality  (r  =.654).  This  result  is  not  particularly  surprising,  given  the  fact  that 
four  out  of  the  six  high-scorers  were  also  extensive  planners,  and  that  two  subjects  in 
this  group  (WS  and  BJ)  each  did  twice  as  much  initial  planning  than  any  other 
subject. 


More  surprisingly,  when  we  looked  at  the  proportion  of  content  plans  in 
relation  to  rhetorical  plans,  we  found  that,  regardless  of  whether  a  'waiter  was  a 
student  or  a  teacher,  the  higher  scorers^O  had  a  lower  percentage  (58.67%)  of  topic 
plans  than  the  lower  scorers  (67. 14%).^  ^  In  fact,  we  found  a  negative  correlation 
between  the  percentage  of  content  planning  and  the  total  quality  score  and  sub-scores 
for  the  final  text  (r  =  -.366  for  total  score;  r  =  -.488  for  audience  score;  r  =  -.398  for 
purpose  score).  While  these  results  are  by  no  means  conclusive,  the  inverse 
relationship  between  the  proportion  of  content  planning  and  text  quality  may  indicate 
that  writers  may  be  focusing  on  planning  specific  content,  and  missing  and/or 
ignoring  important  rhetorical  goals  and  constraints,  such  as  adapting  a  text  for  an 
audience.  For  example,  the  two  writers  who  received  the  lowest  overall  scores  and 
the  lowest  scores  for  audience  and  purpose  (MR  and  GJ;  see  Table  2)  wrote  texts 
which  described  the  technical  aspects  of  their  jobs  in  some  detail,  but  which  were  not 
geared  to  the  young  female  teenage  audience  specified  in  the  assignment. 

These  differences  may  suggest  some  overall  differences  in  the  type  of  initial 
task  representations  that  our  writers  were  building.  The  fact  that  the  "lower  scorers" 
on  our  task  placed  greater  emphasis  on  content  plans  may  suggest  that  at  least  some 
of  them  were  building  a  type  of  "knowledge-telling"  or  topic-oriented  representation 
of  their  task  which  ignored  the  rhetorical  constraints  of  the  assignment.  In  a  task  that 
called  for  planning  to  meet  the  needs  of  the  audience,  some  of  these  writers  seem  not 
to  be  developing  many  rhetorical  goals.  In  contrast,  while  the  "higher  scorers"  do 
devote  about  half  of  their  planning  to  content,  they  are  clearly  placing  more  emphasis 
on  developing  rhetorical  features-or  building  a  "rhetorical  representation"  of  their 
task. 


Second,  as  Table  4  indicates,  we  found  that  all  the  higher  scoring  writers 
developed  plans  in  all  four  rhetorical  categories-i.e.,  theme,  form,  audience  and 
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other  goals—as  well  as  topic  plans  in  their  initial  task  representation.  In  contrast,  only 
one  of  the  lower  scoring  writers  covered  all  these  categories  (and  he  was  a  writing 
teacher);  the  others  ignored  one  or  more  categories  and  the  two  writers  who  received 
the  lowest  text  scores  ignored  several  categories.  The  differences  between  the  scores 
of  the  writers  who  covered  all  categories  in  their  initial  planning  and  those  who 
ignored  one  or  more  categories  was  significant  by  the  Mann-Whitney  test  (p  = 

.0149).  Furthermore,  several  of  the  writers  with  low  quality  scores  failed  to  include 
plans  for  a  "theme"  or  focus  for  the  text.  We  found  that  those  writers  who  included 
plans  in  the  "theme"  category  received  significantly  higher  scores  for  purpose/theme 
on  their  final  texts  (p  =  .0185  by  the  Mann-Whitney  test). 

Thus  we  found  that  the  writers  who  produced  the  best  texts  at  least  "touched 
base"  with  key  rhetorical  features,  even  if  like  expert  BS,  they  did  little  initial 
planning.  The  four  least  successful  novices  (i.e.,  those  writers  with  the  lowest  scores 
for  their  texts)  demonstrated  a  lack  of  concern  for  rhetorical  features,  either  because 
they  did  not  have  the  rhetorical  knowledge  to  deal  with  audience,  form  etc.  or  because 
they  did  not  realize  that  they  should  do  so.  This  question  of  whether  some  writers  fail 
to  plan  rhetorically  because  they  do  not  have  sufficient  knowledge  or  because  they  do 
not  think  to  apply  their  rhetorical  knowledge  will  be  addressed  in  a  further  study. 

For  a  closer  look  at  the  nature  of  the  information  in  our  writers'  initial  task 
representation,  we  will  look  at  some  examples  in  depth.  Below  are  two  planning 
episodes  from  the  beginning  of  the  protocol  of  one  of  our  more  successful  expert 
"extensive"  planners  (WS),  whose  text  was  rated  highest.  Although  these  episodes 
cover  only  about  a  fifth  of  the  task  representation  he  builds  before  beginning  to  write, 
they  do  illustrate  some  important  features  of  his  planning  process.  We  include  the 
actual  protocol  and  an  analysis  of  the  subject's  planning  process. 


Writer  WS;  extensive  planner;  '’higher  scorer  on  text";  expert  writing  teacher 


Clause  # 

Analysis 

Episode  1 

16  Job— English  teacher  rather  than  professor 

Topic 

18  In  fact  that  might  be  a  useful  thing  to 
to  focus  on 

19  ...how  a  teacher  differs  from  a  professor 

20  and  I  see  myself  as  a  teacher 

21  That  might  help  my  audience  to  reconsider 
their  notion  of  what  an  English  teacher 
does 

Theme 

Defines  focus 
of  text 

Goal/theme 
Sets  top-level 
goal 

23  (Reads)  young  female  teenage  audience 

24  They  will  all  have  had  English 

25  Audience-they're  all  in  school 

26  They're  taking  English 

27  For  many  of  ^em  English  may  be  a 
favorite  subject 

Audience 
Represents 
relevant 
background  of 
reader 

29  But  for  the  wrong  reasons-some  of  then 
may  have  the  wrong  reasons  in  that 

English  is  good 

Audience 

Represents 

attitudes 
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30  Because  it's  tidy 

31  Can  be  a  neat  tidy  little  girl 

32  Others  turned  off  because  it  seems  too  prim 

33  By  God,  I  can  change  that  notion  for  them  Goal/theme 

Episode  3 


45  All  right  I'm  an  English  teacher 

47  I  know  they  are  not  going  to  be 

Audience 

to  be  disposed  to  hear  what  I'm  saying 

Represents 

attitudes 

48  Partly  for  that  reason  and  partly  to 

put  them  in  the  right— the  ^nd  of 

Goal 

frame  of  mind  I  want 

(Prepare  audience) 

49  I  want  to  open  with  an  implied 

question  or  a  direct  one 

Form 

50  and  put  them  in  the  middle  of  some 

Develops  skeleton 

situation 

structure  for  text 

5 1  then  expand  from  there  to  talk  about 

my  job  more  generally 

52  and  try  to  tie  it  in  with  their  interest 

Goal 

(Involve  audience) 

53  So  one  question  is  where  to  begin 

Form 

54  Start  in  the  middle  of-probably  the 

Plans  introduction 

first  day  of  class 

55  They'd  be  interested 

56  They'd  probably  clue  into  that  easily 

Audience 

57  because  they  would  identify  with  the 

Represents  a 

first  days  of  school 

shared  reference 

58  and  my  first  days  are  raucous  affairs 

59  It  would  imme^ately  shake  'em  up 

Goal 

60  and  get  them  to  think  in  a  different 

Develops  specific 

context 

audience  goals 

for  introduction 

62  Maybe  first  101  class  with  that  crazy 

Topic 

skit  I  put  on 

Elaborates  text 

plan 

63  That's  probably  better  than  305 

64  because  101  is  freshmen 

Audience 

65  and  that's  nearer  to  their  level 

66  and  that  skit  was  crazy 

67  and  it  worked  beautifully 

68  Okay— let's  see  before  I  write  that 

Process  goal 

69  I  think  I'll  give  myself  some  notion  of 

where  I'm  heading 

(Continues  to  plan  up  to  clause  #135  when  he  ties  out  a  first  sentence 

which  he  rejects.  Initial  task  representation  continues 

to  clause  #201  when  he  comes  up  with  a  first  sentence  which  he  keeps) 
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What  features  do  we  see  in  these  excerpts  which  would  lend  support  to  our 
hypothesis  that  experts  build  a  rhetorical  task  representation? 

First,  this  writer's  task  representation  is  rich  in  rhetorical  information. 

In  these  episodes  he  develops  plans  and  goals  in  all  the  categories  that  we 
hypothesized  are  key  features  of  a  rhetorical  task  representation.  Not  only  does  he 
develop  plans  for  specific  content  to  include  in  his  text,  but  he  also  develops  a  theme, 
a  partid  structure,  a  quite  detailed  representation  of  his  audience  and  a  set  of 
task-specific  goals.  For  example,  he  develops  a  representation  of  the  audience  which 
includes  their  background  (they  take  English  in  school  and  may  enjoy  it)  and  their 
attitudes  (they  are  not  going  to  be  disposed  to  hear  what  I'm  saying).  From  this 
representation  he  develops  a  set  of  goals  for  his  audience-to  "shake  them  up  and  get 
them  to  think  in  a  different  context"  (clauses  59/60). 

Second,  the  goals  that  he  generates  provide  an  integrated  rhetorical 
framework  for  his  planning.  Instead  of  piecemeal  idea  generation  or  brainstorming, 
we  see  this  writer  generating  and  organizing  content  to  meet  his  particular  persuasive 
purpose.  For  example,  he  starts  out  with  a  scenario  which  will  put  the  audience  in 
the  right  frame  of  mind  and  which  will  help  them  to  think  about  English  teaching  in  a 
different  context.  His  text  plans  are  thus  adapted  to  fit  with  his  guiding  focus.  In 
addition,  many  of  his  goals  interact  with  each  other  and  two  goals  may  be  instantiated 
by  a  single  text  plan.  For  example,  he  comes  up  with  the  idea  to  talk  about  his  first 
day  of  class  because  this  would  further  his  goal  to  shake  up  his  audience  and  because 
this  would  be  something  his  readers  could  identify  with.  Thus  this  one  text  plan 
instantiates  two  of  his  important  goals. 

In  sum,  this  writer  is  generating  a  set  of  goals  and  plans  which  make  this  task 
uniquely  his  own.  While  the  task  instructions  provide  him  with  some  loosely  defined 
goals  and  constraints  (the  topic,  the  genre  and  the  audience),  he  uses  the  umbrella  of 
his  rhetorical  representation  to  adapt,  elaborate  and  instantiate  these  goals.  The  task 
that  he  ends  up  doing  is  very  much  a  task  that  he  himself  has  created  and  a  task  which 
results  from  a  coherent  theory  based  on  rhetorical  principles  and  concepts. 

How  does  this  expert's  task  representation  differ  from  that  of  one  of  the  less 
successful  writers?  Below  are  some  early  protocol  episodes  from  one  of  our  less 
successful  student  writers,  whose  text  was  rated  eighth  out  of  twelve. 


Writer  LR:  extensive  planner;  "lower  scorer  on  text";  novice  writer 


Clause  # 

Analysis 

Episode  1 

6  I'm  going  to  assume  here  I'm  an 

Topic 

engineer 

Defines  subject 

8  I  guess  a  research— a  research  engineer 

matter  of  text 

. 

Topic 

10  However,  there's  a  graduate  student 

(to  the  end  of 

1 1  because  I'm  a  graduate  student 

Episode  I) 

12  So  ...really  my  job  is  I'm  going  to  school 

Explores  aspects 

13  Let's  see- I'm  a  graduate  student 

14  and  I'm  an  engineer 

15  and  I'm  a  research  engineer 

of  job 

14 


16  So  everything  is  there 

17  So  maybe  I  should  explain  here  instead 
is  that  I'm  a  graduate  student  pursuing 
a  Ph.D.  in  engineering 

20  I  do  research  work 

22  I  teach  a  course 

Episode  2 

24  (Reads)  for  a  thirteen  to  fourteen  teenage 
audience 

25  So  we  have  to  address  the  fact  that  this 
girl  is  seventh  or  eighth 

27  Okay  so  these  are  all  the  things  going 
through  my  head  relative  to  engineering 
-research  engineering 

28  These  are  all  die  things  I'm  supposedly  doing 

29  and  I'm  pursuing  a  Ph.D. 

30  Working  on  a  thesis 

32  My  job  here  is  wrong,  the  way  I'm 
interpreting  my  job 

34  The  way  I'm  interpreting  my  job  means  what 
I'm  doing  with  my  life  at  this  moment 
in  time 

Episode  3 

37  For  the  girl  is  approximately  in  seventh 
or  eighth  grade 

38  ...the  assignment  has  to  appeal  to  a  broad 
range  in  intellect 

39  It  must  explain  simply  what  I  am  doing 

43  I  have  to  generate  an  essay 

44  We'll  assume  it's  about  two  pages 


48  That's  not  really  hard 

49  I  really  have  my  first  line 

50  so  I'm  going  to  rip  off  the  page  here 
52  I  think  I  can  write  this  out  very  quickly. 

(Tries  out  first  sentence  at  clause  #53  which  he  rejects; 
generates  first  sentence  which  he  keeps  at  clause  #66) 

This  novice's  task  representation  looks  different  from  the 
expert's  in  several  ways: 


Audience 


Topic 

Reviews  possible 
details  to  include 


Topic 

Re-defines  topic 


Goals 
Sets  general 
audience  goals 

Form 

Specifies  genre 
and  length. 


Process 
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First  the  subject  generates  few  rhetorical  goals  in  contrast  to 

the  expert's  rich  rhetorical  planning.  Rather  most  of  the  planning 

is  related  to  features  of  the  topic--e.g.,  the  main  aspects  of  his  \ 

job  as  a  graduate  student  in  engineering  and  whether  being  a 

researcher  is  in  fact  his  job.  He  includes  no  goals  for  an  overall 

theme  or  focus  to  the  text;  little  information  about  organization  \ 

beyond  a  very  general  sense  of  the  genre  and  the  length;  almost  no 

information  on  his  audience's  interests  or  characteristics,  beyond  i 

that  provided  in  the  task  instructions;  and  no  goals  for  the  text 

beyond  a  rather  vague  sense  that  he  should  "appeal  to  a  broad  range 

in  intellect"  and  that  the  text  "must  explain  simply  what  (he  is)  doing" 

(Clauses  38/39).  ' 

The  very  general  rhetorical  goals  that  he  does  come  up  with  (e.g.,  address  the 
fact  that  this  girl  is  in  the  7th  or  8th  grade)  are  not  further  instantiated  with  sub-goals 
and  text  plans  and  he  is  unable  to  build  upon  them. 

Second,  we  do  not  see  the  integrated  overall  framework  to  guide  his  planning 
process.  His  planning  seems  to  be  generated  haphazardly  by  features  of  the 
content  with  which  he  is  struggling  (i.e.,  a  definition  of  his  job),  rather  than 
integrated  around  rhetorical  features.  Unable  to  transform  his  content  to  meet  the 
constraints  of  the  assignment  and  unable  to  build  his  own  unique  set  of  rhetorical 
goals  for  his  task,  this  writer  has  little  option  but  to  "knowledge  tell."  His  text 
reflects  this  limitation  as  he  produces  a  rather  dry  description  of  energy  research 
which  our  raters  judged  to  be  low  in  meeting  the  needs  of  the  audience.  (See  Table  2 
for  text  scores.) 

In  sum,  this  writer's  planning  does  not  allow  him  to  develop  a  rhetorical 
focus;  eventually,  after  struggling  for  some  time  with  his  topic  knowledge,  he  simply 
plunges  into  writing  the  text,  letting  his  topic  information  drive  his  planning  and 
generating  process.  Although  he  does  plan  more  extensively  than  many  of  our  other 
subjects,  his  plans  do  not  help  him  to  carve  out  a  very  appropriate  and  effective 
representation  of  his  task. 

Although  we  can  see  limitations  in  this  novice  writer's  task  representation,  he 
does  produce  a  text  which  was  rated  higher  than  those  of  two  of  our  novices  whom 
we  characterized  as  "minimal"  planners.  TTiese  writers  (MR  and  GJ;  see  Tables  1  and 
4)  in  fact  did  almost  no  initial  planning  at  all  and  simply  plunged  into  writing 
immediately.  As  Table  4  indicates,  these  subjects'  plans  were  mainly  limited  to 
content  plans,  which  amounted  to  a  very  brief  description  of  their  job.  Lacking  any 
rhetorical  goals,  these  writers  had  little  choice  but  to  knowledge  tell,  a  strategy  which 
is  reflected  by  their  low  scores  for  audience  and  purpose  on  their  written  texts. 

IV.  SUMMARY  AND  CONCLUSIONS 

This  exploratory  analysis  of  our  writers'  initial  protocol  episodes  suggests 
that  both  the  quantity  and  qu^ity  of  a  writer's  initial  planning  may  make  a 
surprisingly  large  difference  in  ^e  quality  of  the  final  text,  in  particular  for  writing 
tasks  where  the  overall  goals  are  only  loosely  specified  (such  as  our  experimental 
task).  More  specifically,  our  analysis  has  resulted  in  several  hypotheses; 
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•  Writers  vary  greatly  in  the  amount  of  planning  they  do  before  beginning  to  write. 
Expert  writers  do  not  necessarily  do  more  initial  planning  than  novice  writers. 

•  There  was  no  significant  expert/novice  difference  in  the  amount  of 
initial  planning. 

•  The  quantity  of  planning  that  a  writer  does  before  writing  may  effect  the 
quality  of  the  text  he  or  she  produces:  the  more  initial  planning,  the  better 
the  final  text.  In  general,  "minimal"  planners  are  likely  to  produce  worse 
texts  than  "extensive"  planners,  particularly  in  the  case  of  inexperienced 
writers. 

•  We  found  a  positive  correlation  between  amount  of  planning  and 
text  quality:  r  =  .665,  all  subjects;  r  =  .694,  experts;  r  =  .799, 
novices. 

•  Although  we  found  no  significant  difference  at  the  .05  level 
between  the  total  scores  of  our  extensive  and  minimal  planners 
(p  =  .0782),  we  believe  that  this  hypothesis  warrants  further 
testing  with  more  subjects. 

•  The  quality  of  a  writer's  initial  planning  (in  particular,  planning  to  meet 
the  needs  of  the  audience,  to  develop  a  purpose  and  a  structure)  may  well 
affect  the  quality  of  the  writer’s  text.  Plans  for  the  audience  and  purpose 
are  better  predictors  of  success  than  plans  for  structure. 

•  We  found  a  strong  positive  correlation  between  quality  of  our 
writers'  initial  planning  and  the  total  quality  scores  for  their  texts 
(r  =  .874). 

•  We  found  a  strong  positive  correlation  between:  1)  quality  of 
audience  plans  and  appropriateness  of  the  text  for  the  audience 
(r  =  .778);  and  2)  between  quality  of  plans  for  purpose  and 
purpose  score  for  the  text  (r  =  .741).  The  correlation  between 
quality  of  plans  for  structure  and  text  score  for  structure  was 
somewhat  lower  (r  =  .489). 

•  Writers  who  produce  successful  texts  appear  to  do  proportionately  less 
"content  planning"  in  their  initial  task  representation,  than  do  those  who 
produce  less  successful  texts;  conversely,  more  successful  writers' 
initial  plans  contain  proportionately  more  rhetorical  goals  for  audience, 
theme,  form  and  for  features  such  as  tone  and  style.  However,  "better" 
writers  may  do  more  content  planning  in  terms  of  absolute  numbers 
because  they  are  more  likely  to  do  more  planning  of  all  types— i.e.,  be 
"extensive  planners."  Our  results,  while  by  no  means  conclusive,  suggest 
that  less  successful  writers'  emphasis  on  "content  planning"  (as  opposed 
to  rhetorical  planning)  results  in  texts  which  are  less  well  adapted  to  the 
audience  and  which  do  not  manifest  a  clear  rhetorical  purpose  or 
organizational  structure-that  is,  texts  weaker  on  rhetoric^  as  opposed  to 
content  features. 
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•  We  found  that  our  higher  scorers  did  more  content  planning  in 
terms  of  absolute  numbers  (159  els.  as  compared  with  47  els.), 
though  this  difference  was  not  significant.  There  was  a  positive 
correlation  between  number  of  content  plans  and  text  quality 

(r  =  .654).  (This  is  explained  by  the  fact  that  four  out  of  six  of  our 
high  scorers  were  extensive  planners,  and  two  did  twice  as  much 
planning  (each)  than  any  other  subject). 

•  We  found  that  those  writers  whose  texts  received  scores  in  the  top 
half  of  the  distribution  of  final  scores  had  a  lower  percentage 
(58.67%)  of  content  planning  than  did  those  writers  whose  scores 
fell  in  the  lower  half  of  the  distribution  (67.14%).  Although  these 
figures  were  not  statistically  significant  (p  =  .06^),  we  believe 
that  this  difference  is  sufficiently  large  to  warrant  further 
investigation. 

•  We  also  found  a  negative  correlation  between  the  percentage  of 
content  planning  and  the  quality  scores  for  the  final  text;  r  =  -.366, 
total  score;  r  =  -.488,  audience;  r  =  -.398  purpose. 


•  Writers  who  produce  more  successful  texts  are  likely  to  plan  in  all  the 
rhetorical  categories  discussed  earlier  when  building  their  initial  task 
representation,  whereas  writers  with  less  successful  texts  are  likely 
to  ignore  some  categories. 

•  We  found  that  the  differences  between  the  text  scores  of  the 
writers  with  plans  in  all  our  rhetorical  categories  and  those  with 
one  or  more  categories  missing  was  significant  (p  =  .0149). 

•  Those  writers  who  included  plans  for  a  theme  or  focus  for  their 
text  received  significantly  higher  scores  than  those  who  did  not 
(p  =  .0185). 

•  Writers  who  produce  more  successful  texts  (in  our  case,  not  necessarily 
the  expert  writing  teachers)  build  a  rhetorical  representation  of  their  task 
which  functions  as  a  coherent  theoiy  of  the  task.  Less  successful  writers 
often  do  not  (or  perhaps  cannot)  build  a  rhetorical  representation  of  their 
task  and  may  be  left  with  an  ineffective  knowledge-telling  strategy  which 
ignores  many  of  the  goals  and  constraints  of  the  assignment.  We  defined 
rhetorical  representations  to  be: 

•  rich  in  rhetorical  goals-all  the  rhetorical  categories  of  planning  are 
covered, 

•  integrated  within  a  rhetorical  framework,  and 

•  unique  for  the  particular  task  at  hand. 


This  exploratory  study  has  highlighted  the  importance  of  initial  planning  as  a 
possible  predictor  of  text  quality.  The  more  successful  writers  simply  did  more  initial 
planning,  and  they  did  a  qualitatively  different  kind  of  planning  with  an  emphasis  on 
rhetorical  features.  We  are  currently  developing  studies  to  further  investigate  these 
differences  and  to  address  some  issues  raised  by  this  study.  For  example,  are  novice 
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writers  unable  to  develop  extensive  initial  plans  and  to  plan  rhetorically  because  they 
lack  the  knowledge  to  do  so,  or  is  their  apparent  failure  a  matter  of  executive 
control— i.e.,  are  they  failing  to  plan  because  thv^y  do  not  prompt  themselves 
(or  "think")  to  do  so?  Certainly  research  on  young  writers'  cognitive  processes 
(Bereiter  and  Scardamalia,  1982)  indicates  that  failure  of  executive  control  is  a  major 
stumbling  block  for  many  developing  writers.  Our  follow-up  study  presents  student 
writers  with  a  series  of  prompts  designed  to  elicit  "expert  planning"  behavior  and 
should  provide  some  further  insights  into  a  theory  of  expert  and  novice  planning 
strategies. 
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FOOTNOTES 


I  We  defined  the  "first  sentence"  as  the  first  sentence  which  the  writer  built  upon  to 
generate  a  draft  or  a  partial  draft  (even  if  that  draft  was  changed  later).  Thus 
sentences  generated  and  immediately  abandoned  were  not  taken  as  a  "first  sentence" 
but  as  a  note. 

^  See  Appendix  B  for  a  more  detailed  explanation  of  this  coding  scheme. 

3  We  should  note  here  that  because  seven  out  of  our  twelve  subjects  were  novices 
we  would,  of  course,  expect  one  novice  to  obtain  a  text  score  above  the  median  and 
to  fall  into  our  extensive  planning  category.  In  fact  three  novices  received  text  scores 
in  the  top  half  of  the  distribution  of  scores  and  three  were  classified  as  "extensive” 
planners. 

^  We  used  to  median  split  on  the  total  number  of  planning  clauses  (excluding 
repetitions)  in  the  initid  segments  of  our  subjects'  protocols  (i.e.,  up  to  their 
generation  of  a  first  sentence)  to  distinguish  these  two  "modes"  of  planning. 
Extensive  planners  were  those  in  the  top-half  of  the  distribution  of  the  number  of 
planning  clauses;  minimal  planners  were  those  in  the  lower  half  of  the  distribution. 
(See  Table  1.) 

5  The  scores  in  Table  2  represent  the  sums  of  the  scores  arrived  at  by  four  raters  on 
three  dimensions:  appropriateness  for  the  audience,  rhetorical  purpose  and  structure. 
Inter-rater  reliability  on  these  scores  was  .40  on  average.  Individual  figures  were 
.619,.243,.l  16,.375,.536,.451.  While  these  figures  are  in  some  cases  rather  low, 
they  seem  to  reflect  a  general  difficulty  in  obtaining  reliability  on  overall  ratings  for 
written  texts.  See  Appendix  A  for  a  copy  of  the  rating  instructions. 

6  As  noted  earlier,  our  rating  criteria  emphasized  rhetorical  features  of  the  text.  We 
suspect  that  our  experts  who  received  relatively  low  scores  for  their  texts  may  have 
done  so  because  they  were  not  able  to  adapt  a  discussion  of  their  job  for  the  young 
audience  in  the  assignment  as  effectively  as  some  of  the  student  writers. 

^  The  scores  are  the  sum  of  two  ratings  on  three  dimensions:  audience,  purpose  and 
organization.  Inter-rater  reliability  on  the  total  scores  was  .807  by  the  Pearson 
product  moment  correlation  co-efficient.  The  rating  instructions  are  in  Appendix  A. 

8  Inter-coder  reliability  on  the  coding  was  71.4%. 

^  We  performed  a  median  split  on  the  text  quality  scores  to  obtain  these  two  groups. 
As  there  were  seven  novices  and  five  experts  we  would  expect  one  novice  to  be 
placed  in  the  "higher  scorers"  group;  in  fact  three  received  scores  in  the  top  half  of  the 
distribution  of  scores. 

When  we  looked  at  our  writers  in  terms  of  expert/novice  differences,  we  did  find 
that  on  average  the  experts  (i.e.,  the  writing  teachers)  also  had  a  lower  percentage  of 
content  planning  (51.4%)  than  did  the  student  writers  (73.7%). 

I I  These  figures  were  not  statistically  significant,  though  the  scores  for  audience 
approached  significance  by  the  Mann-Whitney  text  (p  =  .0631). 
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Subject 

Total  Clauses 

Total  Planning 
excluding  repetitions 

Planning 

Mode 

Experts 

WS 

201 

88 

Extensive 

BJ 

209 

83 

Extensive 

BS 

24 

7 

Minimal 

KC 

46 

24 

Extensive 

IB 

23 

2 

Minimal 

Novices 

NK 

89 

44 

Extensive 

LR 

67 

21 

Extensive 

SA 

99 

38 

Extensive 

SE 

28 

11 

Minimal 

MR 

8 

4 

Minimal 

FD 

28 

11 

Minimal 

GJ 

15 

8 

Minimal 

Table  1.  Amount  of  Planning  before  Beginning  to  Write 

*  Units  are  clauses. 
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Planning 

Mode 

Audience 

Purpose 

Structure 

Total 

Experts 

WS 

ext. 

15 

14 

12 

41 

BJ 

ext. 

14 

12 

13 

39 

BS 

min. 

9 

13 

13 

35 

KC 

ext. 

11 

7 

9 

27 

IB 

min. 

13 

10 

11 

34 

Novices 

NK 

ext. 

12 

14 

14 

40 

LR 

ext. 

8 

9 

12 

29 

SA 

ext. 

12 

13 

11 

36 

SE 

min. 

15 

12 

9 

36 

MR 

min. 

6 

6 

10 

22 

FD 

min. 

14 

7 

4 

25 

GJ 

min. 

4 

4 

11 

19 

Table!.  Quality  Scores  for  Texts* 

*  Scores  arc  the  sum  of  four  raters  who  rated  the  texts  on  three  dimensions 
(audience,  purpose  and  structure).  Each  dimension  was  rated  on  a  four  point  scale 
(l=lowest;4=highest).  Highest  possible  score  =  48  ;  lowest  possible  score  =  12. 
Inter-rater  reliability  on  these  figures  was  an  average  of  .4  (highest  pair  =  .619)  by 
the  Pearson  product  moment  correlation  co-efficient. 
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Planning 

Mode 

Experts 

Audience 

Purpose 

Structure 

Total 

Protocol 

Total 

Text** 

WS 

ext. 

8 

8 

8 

24 

41 

BJ 

ext. 

8 

6 

7 

21 

39 

BS 

min. 

5 

4 

5 

14 

35 

KC 

ext. 

5 

4 

4 

13 

27 

IB 

Novices 

min. 

4 

2 

2 

8 

34 

NK 

ext. 

8 

7 

7 

22 

40 

LR 

ext. 

4 

4 

6 

14 

29 

SA 

ext. 

5 

4 

8 

17 

36 

SE 

min. 

8 

5 

5 

18 

36 

MR 

min. 

2 

2 

2 

6 

22 

FD 

min. 

4 

4 

3 

11 

25 

GJ 

min. 

2 

2 

2 

6 

19 

Table  3.  Quality  Scores  for  Initial  Protocol  Excerpts* 


*  Scores  are  the  sum  the  scores  of  two  raters  who  rated  the  protocols  on  three  dimensions 
(audience,  purpose  and  structure).  Each  dimension  was  rated  on  a  four  point  scale 
(l=lowest;  4=highest).  Highest  possible  total  score  =  24;  lowest  possible  total  =  6.  Inter-rater 
reliability  cn  these  scores  was  .807  (on  total  scores)  by  Pearson  product  moment  correlation 
coefficient. 

**  The  breakdown  of  these  total  text  scores  is  presented  in  Table  2.  The  correlation 
between  total  quality  scores  on  the  protocols  and  total  quality  scores  on  the  text  was  .874. 


Subject  Total* 

High  Scorers*** 

Topic 

Theme** 

Form 

Audience 

Other 

Goals 

WS  (Expert) 

88 

41 

5 

17 

13 

18 

BJ  (Expert) 

83 

46 

3 

25 

3 

9 

BS  (Expert) 

7 

3 

2 

1 

2 

1 

NK  (Novice) 

44 

35 

3 

4 

3 

2 

SA  (Novice) 

38 

29 

1 

3 

1 

4 

SE  (Novice) 

11 

5 

1 

2 

1 

2 

TOTAL 

271 

159 

15 

52 

23 

36 

%  of  total 

Low  Scorers 

58.7% 

5.5% 

19.2% 

8.5% 

13.3% 

KC  (Expert) 

24 

13 

2 

3 

6 

2 

IB  (Expert) 

2 

2 

- 

- 

- 

LR  (Novice) 

21 

15 

- 

3 

1 

2 

MR  (Novice) 

4 

3 

- 

1 

- 

- 

FD  (Novice) 

11 

6 

- 

1 

3 

1 

GJ  (Novice) 

8 

8 

- 

- 

- 

- 

TOTAL 

70 

47 

2 

8 

10 

5 

%  of  total 

67.1% 

2.8% 

11.4% 

14.3% 

7.1% 

Table  4.  Initial  Task  Representation;  Types  of  Planning 

*  Total  planning  clauses  excluding  repetitions. 

Clauses  coded  as  theme  were  also  coded  in  another  category  so  percentages  total 
more  than  100%. 

sfc  sfc 

Subjects  with  scores  in  the  top  half  of  the  distribution. 
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APPENDIX  A 


Rating  Instructions  for  Texts 

The  attached  texts  were  produced  by  twelve  writers  who  were  given  the 
following  assignment: 

Describe  your  job  for  a  young  teenage  female  audience  (ages  13-14). 

Your  text  will  appear  in  Seventeen  Magazine. 

As  you  will  see,  some  of  the  texts  were  written  by  students  and  some  by 
teachers.  We  are  interested  in  your  judgments  about  the  quality  of  the  texts, 
independent  of  whether  the  writer  is  a  student  or  a  teacher. 

Please  rate  the  quality  of  the  texts  according  to  three  criteria:  audience, 
purpose  and  structure.  (A  more  detailed  explanation  of  these  criteria  is  given 
below.)  Each  text  should  receive  three  scores,  each  on  a  four-point  scale. 

Please  consider  each  of  the  three  ratings  on  separate  passes-that  is,  rate  all 
the  texts  for  audience,  then  all  the  texts  for  purpose,  and  then  all  the  texts  for 
structure.  Some  texts  may  be  good  on  one  dimension,  but  not  on  others. 

In  many  cases  raters  tend  to  be  conservative  in  their  judgments  and  use  only 
the  middle  points  of  the  scale.  Here  we  would  like  you  to  spread  your  judgments 
over  the  whole  range  of  the  scale.  (We  would  really  like  to  make  some  clear 
distinctions  between  the  relative  quality  of  these  texts  on  the  three  rating 
dimensions.) 


Rating  Criteria 

1.  How  well  is  the  text  adapted  for  the  audience  specified  in  the  assignment? 
1  2  3  4 


Not  adapted 
at  all  for  audience 


Very  well  adapted 
for  audience 


2.  To  what  extent  does  this  text  have  a  clear  point,  focus  or  rhetorical 
purpose  that  goes  beyond  simply  "knowledge-telling"  on  a  topic? 


1 

No  clear  point/ 
focus/purpose 


Very  clear  point/ 
focus/purpose 


How  well-structured  is  the  text?  Consider  both  overall  organization  and 
coherence. 

1  2  3  4 

Not  Very  well-structured 

well-structured 
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APPENDIX  B 


Rating  Instructions  for  Protocol  Excerpts 

The  attached  protocol  excerpts  contain  the  initial  plans  (i.e.,  plans  generated 
before  writing  a  first  sentence*)  of  twelve  writers  who  were  given  the  following 
assignment: 

Describe  your  job  for  a  young  female  teenage  audience  (ages  13-14). 

Your  text  will  appear  in  Seventeen  Magazine. 

As  you  will  see,  some  of  the  writers  were  students  and  some  were  teachers.  In 
addition,  some  of  our  subjects  did  most  of  their  planning  before  beginning  to  write, 
whereas  others  did  most  of  their  planning  during  the  process  of  generating  a  draft.  Here 
we  are  interested  in  your  judgments  about  the  quality  of  the  planning  in  these  initial 
protocol  excerpts,  independent  of  whether  the  writer  was  a  student  or  a  teacher. 

Please  rate  the  quality  of  the  writers'  planning  in  these  protocol  excerpts 
according  to  three  criteria:  audience,  purpose  and  structure.  (These  criteria  are  explained 
in  more  detail  below.)  Each  excerpt  should  receive  three  scores,  each  on  a  four-point 
scale. 


Please  consider  each  of  the  three  ratings  on  separate  passes  —  that  is,  rate  all  the 
excerpts  for  audience,  then  all  the  excerpts  for  purpose  and  then  all  the  excerpts  for 
structure.  Some  of  the  excerpts  may  be  good  on  one  dimension,  but  not  on  others. 

In  many  cases  raters  tend  to  be  conservative  in  their  judgments  and  to  use  only  the 
middle  points  on  the  scale.  Here  we  would  like  you  to  spread  your  judgments  over  the 
whole  range  of  the  scale.  (We  would  really  like  to  make  some  clear  distinctions  between 
the  relative  quality  of  the  writers'  planning  on  the  three  rating  dimensions.) 


Rating  Criteria 

1 .  How  well  does  the  writer's  planning  reflect  a  concern  for 
adapting  his/her  text  for  the  audience  specified  in  the  assignment? 

1  2  3  4 


No  concern 
with  adapting 
to  the  audience 


Very  concerned  with 
adapting  to  the 
audience 


2.  To  what  extent  does  the  writer's  planning  reflect  a  concern  with  developing  a 
clear  point,  focus  or  rhetorical  purpose  for  his/her  text  that  goes  beyond 
simply  "knowledge-telling"  on  a  topic? 

1  2  3  4 

No  concern  Very  concerned  with 

with  point/  point/focus/purpose 

focus/purpose 


27 


3.  How  far  does  the  writer's  planning  reflect  a  concern  with  structuring  the  text 
or  fitting  a  genre? 


1  2  3 

No  concern 
with  structure 


4 

Very  concerned  with 
structure 


Note:  These  excerpts  end  at  the  point  where  the  subjects  wrote  their  first  sentence  of 
text.  (For  the  purpose  of  this  analysis,  the  "first  sentence"  was  defined  as  the  first 
sentence  which  the  writer  built  upon  to  generate  a  first  draft.  Sentences  which  were 
generated  and  then  immediately  rejected  were  considered  "notes"  rather  than  text.) 
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